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CLAIMS 

What is claimed is: 

1 . A method for controlling a computer using a video image, the method 
comprising: 

5 (a) capturing a video stream, the video stream comprising a plurality 

of video frames; 

(b) determining a location of an object in the video stream in at least 
some of the plurality of video frames; 

(c) controlling a program executing on the computer based on the 
1 0 location of the object; 

(d) combining the video stream with a user interface stream 
generated by the computer operating system, thereby forming a 
composite video stream; and 

(e) displaying the composite video stream. 

15 2. The method of claim 1 wherein capturing a video stream includes 
receiving a live video signal of a user generated by a video camera. 

3. The method of claim 1 wherein capturing a video stream includes 
receiving a stored video signal from a video storage device. 

4. The method of claim 1 wherein determining the location an object in the 
20 video stream includes: 

(a) searching for a predetermined color in the video stream; 

(b) in response to locating the predetermined color, identifying an 
occurrence of the predetermined color having the largest area; 
and 

25 (c) determining coordinates of the center of the occurrence of the 

predetermined color having the largest area. 

5. The method of claim 1 wherein controlling a program executing on the 
computer based on the location of the object: 

(a) analyzing motion of the object in successive video frames to 
30 determine presence of a control event; and 

(b) controlling the program based on the control event. 
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6. The method of claim 5 wherein the video stream comprises an image of 
a computer user, the object comprises an object associated with the 
user's hand, and the control event comprises a pointer movement event. 

7. The method of claim 5 wherein the video stream comprises an image of 
5 a computer user, the object comprises an object located in the user's 

hand, and the control event comprises a mouse click event. 

8. The method of claim 1 wherein combining the video stream with the 
user interface stream generated by the computer operating system 
includes horizontally reversing frames of the video stream to produce a 

1 0 mirror image of the frames of the video stream. 

9. The method of claim 1 wherein combining the video stream with the 
user interface stream generated by the computer operating system 
includes transparently overlaying the user interface stream on the video 
stream. 

15 10. The method of claim 1 wherein combining the video stream with the 
user interface stream generated by the computer operating system 
includes transparently overlaying the video stream on the user interface 
stream. 

11. The method of claim 1 wherein combining the video stream with the 
20 user interface stream generated by the computer operating system 

includes: 

(a) adjusting a transparency level of at least one of the user interface 
stream and video stream; and 

(b) generating the composite stream from the user interface stream 
25 and the video stream. 

12. The method of claim 11 wherein adjusting the transparency level 
includes dynamically adjusting the transparency level. 

13. The method of claim 1 wherein displaying the composite video stream 
includes projecting the composite video stream. 
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14. The method of claim 1 comprising combining a plurality of different 
video streams with the user interface stream and displaying the different 
streams with the user interface stream. 

15. The method of claim 14 wherein at least some of the video streams 
5 originate from stored video signals. 

16. A method for combining a video image of a user with a computer 
desktop interface, the method comprising: 

(a) capturing a video stream of a user, the video stream comprising 
a plurality of video frames; 

10 (b) transparently combining the video stream with a computer 

desktop generated by the computer operating system, thereby 
forming a composite video stream; and 
(c) displaying the composite video stream, wherein the composite 
image includes a transparent image of the user displayed with 

1 5 the computer desktop. 

1 7. The method of claim 1 6 wherein capturing the video stream of the user 
includes receiving a live video signal generated by a video camera. 

18. The method of claim 16 wherein combining the video stream with the 
user interface stream generated by the computer operating system 

20 includes horizontally reversing frames of the video stream to produce a 

mirror image of the frames of the video stream. 

19. The method of claim 16 wherein combining the video stream with the 
user interface stream generated by the computer operating system 
includes: 

25 (a) adjusting a transparency level of at least one of the user interface 

stream and video stream; and 

(b) generating the composite stream from the user interface stream 
and the video stream. 

20. The method of claim 19 wherein adjusting the transparency level 
30 includes dynamically adjusting the transparency level. 
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21. The method of claim 16 wherein displaying the composite video stream 
includes projecting the composite video stream. 

22. The method of claim 1 6 wherein displaying the composite video stream 
includes displaying the composite video stream on a non-projection 

5 computer display device. 

23. The method of claim 1 6 wherein displaying the composite video stream 
includes displaying a mirror image of the user with the desktop. 

24. The method of claim 1 6 comprising controlling objects on the desktop in 
response to movement of the user image. 

10 25. The method of claim 23 wherein controlling objects on the desktop 
includes moving objects on the desktop. 

26. The method of claim 23 wherein controlling objects on the desktop 
^ includes activating programs associated with objects on the desktop. 

27. The method of claim 1 6 wherein the desktop comprises the desktop of a 
1 5 computer local to the user. 

28. The method of claim 1 6 wherein the desktop comprises the desktop of a 
computer remote from the user. 

29. The method of claim 16 comprising transparently combining and 
displaying a plurality of video streams with the computer desktop, 

20 wherein each of the plurality of video streams includes an image of a 

different user. 

30. The method of claim 28 comprising controlling desktop objects in 
response to movement of user images in any of the video streams. 

31. A computer-readable storage medium containing a set of computer- 
25 executable instructions, the set of instructions comprising: 

(a) a video stream capturing routine for capturing a video stream, the 
video stream comprising a plurality of video frames; 

(b) a video frame analysis routine for determining a location of an 
object in at least some of the plurality of video frames; 

30 (°) a driver for controlling a program executing on the computer 

based on the location of the object; 
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(d) a video compositing routine for combining the video stream with 
a user interface stream generated by the computer operating 
system, thereby forming a composite video stream; and 

(e) a video display routine for displaying the composite video stream. 
5 32. The computer-readable storage medium of claim 30 wherein the user 

interface driving routine comprises: 

(a) instructions for searching for a predetermined color in the video 
stream; 

(b) instructions for identifying an occurrence of the predetermined 
1 0 color having a largest area; and 

(c) instructions for determining the coordinates of the center of the 
occurrence of the predetermined color having the largest area. 

33. The computer-readable storage medium of claim 31 wherein the video 
compositing routine comprises: 

1 5 (a) instructions for adjusting the transparency level of at least one of 

the user interface stream and the video stream; and 
(b) instructions for generating the composite stream from the user 
interface stream and the video stream. 

34. The computer-readable storage medium of claim 30 wherein the video 
20 compositing routine comprises instructions for horizontally reversing 

images of the video stream to produce a mirror image of the images of 
the video stream. 

35. A computer program product comprising computer-executable 
instructions embodied in a computer readable medium for performing 

25 steps comprising: 

(a) receiving a video image of a computer user; 

(b) combining the video image of the computer user with a computer 
desktop image; 

(c) displaying the combined image; 

30 (d) tracking a portion of the user image in the combined image; and 
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(e) manipulating objects in the desktop image based on the tracked 
portion. 

36. The computer program product of claim 35 wherein manipulating objects 
includes highlighting the objects. 
5 37. The computer program product of claim 35 wherein manipulating objects 
includes moving the objects. 

38. The computer program product of claim 35 wherein manipulating objects 
includes activating programs associated with the objects. 

39. A computer system comprising: 
10 (a) a display device; 

(b) a video camera for producing a video stream of a user; and 

(c) a processing unit operatively coupled to the display device and 
video camera, wherein the processing unit is adapted to: 

(i) receiving the video stream of user, the video stream 
1 5 comprising a plurality of video frames; 

(ii) determine a location of a predetermined object associated 
with the user some of the plurality of video frames; and 

(iii) control execution of a program based on the location of 
the object. 

20 



